AN IMAGING DEVICE AND A MONITORING SYSTEM 



This application is based on patent application 
No. 2003-067119 filed in Japan, the contents of which 
are hereby incorporated by references. 

BACKGROUND OF THE INVENTION 

1. Field of The Invention 

[0001] The present invention relates to an imaging 

device such as a video camera and a monitoring system 
for detecting and pursuing an object intruded into a 
monitored area. 

2. Related Art Statement 

[0002] A monitoring system is known which uses a 

camera for continuously viewing an area or a scene to 
be guarded, secured or monitored (hereinafter 
referred to as a monitored are). It is desirable 
that an image of a particular object such as an 
intruder in the area is displayed and/or memorized in 
some detail. On the other hand, it is also desirable 
that a relatively large or wide area is monitored for 
the monitoring. When a picture of the area is taken 
through an objective lens having a relatively wide 
field of view, images of the objects in the area are 
small in size . 
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[0003] Japanese Unexamined Patent Publication No. 

5-232208 discloses an e lec t ro - op t ical device for 
taking a picture of a monitored area through an 
optical system which forms an telephoto or magnified 
image in a central portion and distorted image in the 
peripheral area of the image formed by the objective 
lens to ensure wide field of view by the peripheral 
area. The device is arranged to track an object as 
an intruder such that the object is captured in the 
central portion view when the object appears in the 
field of view. 

[0004] However, the prior art device as mentioned 

above displays an image distorted largely in the 
peripheral area so that the image to be displayed is 
inferior in visibility and gives a sense of 
incongruity to human vision. 

[0005] Japanese Unexamined Patent Publication No. 

2000-341568 discloses an image sensing device wherein 
an original image is formed on a CCD by a convex 
mirror called as a fovea mirror having an optical 
characteristic similar to a fovea of a human eye. 
The fovea mirror forms an image with telephoto effect 
in the central area of the image ensuring wide field 
of view by the peripheral area, with the image being 
distorted in the peripheral area. The prior art 

device applies pixel position conversion to image 
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data of the original image to generate a panorama 
image with the distortion being corrected or removed. 
[0006] However, in the second prior art device, 

the distortion of the image around the high- 
resolution image is corrected based on the high- 
resolution image so that an area ratio of the high- 
resolution image to the panorama image is relatively 
small. Thus, when the panorama image is displayed, 
for example, on a display, it is difficult to observe 
this object in detail even if a specific object is 
sensed with a high-resolution image. 

[0007] Another prior art device is known which 

employs a fisheye lens to take a picture of an object, 
and which forms a part of the image formed by the 
fisheye lens, with the part being extracted from the 
entire image. However, it is difficult to observe a 
specific object in detail since the obtained image 
does not have a high resolution over the entire area. 

SUMMARY OF THE INVENTION 

[0008] Accordingly. an object of the present 

invention is to provide an imaging device with which 
a specific object can be visually confirmed in a 
satisfactory manner. 

[0009] Another object of the present invention is 

to provide an imaging device which provides image 
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data representing an image of a specific object with 
a larger scale and good visibility. 

[0010] still another object of the present 

invention is to provide an imaging device which 
operates in a stand-by mode for providing image data 
of a wide area image, and in a close-observation mode 
for providing image of a central area image tracking 
a specified object to capture it in the central area. 
The central area image is obtained by extracting a 
central portion of an image formed by an optical 
system of the imaging device. 

[0011] Further object of the present invention is 

to provide a monitoring system with which a specific 
object can be visually confirmed in a satisfactory 
manner . 

[0012] Still further object of the present 

invention is to provide a monitoring system which 
displays an image of a specific object with a larger 
scale and good visibility. 

[0013] Yet further object of the present invention 

is to provide a monitoring system which operates in a 
stand-by mode for displaying an image of a wide area 
of a monitored region, and in a close - observation 
mode for displaying a image of a central portion of 
the wide area image, tracking a specified object to 
capture it in the central area. 
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[0014] To attain one or more of the objects 

mentioned above, according an aspect of the present 
invention, an imaging device comprises an optical 
system having an optical characteristic that 
distortion is larger in a peripheral area than in a 
central area of the image formed by the optical 
system; an image data generating section for 
generating image data in a stand-by mode for waiting 
for intrusion of an object, and in a close- 
observation mode for taking a picture of the object 
while tracking the object; and a first image data 
processing section for generating, in the close- 
observation mode, a central image data representing 
an image of the central area, with the central image 
data being extracted from the image data generated by 
the image data generating section. 

[0015] According to another aspect of the present 

invention, a monitoring system comprises a imaging 
device for generating image data representing an 
image of a central area of an image formed by an 
optical system, a controller including a display; and 
a communicating section for enabling communication 
between the imaging device and the controller, the 
display of the controller displaying the image of the 
central area when the central image data is 
transmitted from the imaging device to the controller 
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through the communicating section. The optical 

system has an optical characteristic that distortion 
is larger in a peripheral area than in a central area 
of the image formed by the optical system. The 
imaging device includes an image data generating 
section for generating image data in a stand-by mode 
for waiting for intrusion of an object. and in a 
close-observation mode for taking a picture of the 
object while tracking the object; and a first image 
data processing section for generating, in the close- 
observation mode, a central image data representing 
an image of the central area, with the central image 
data being extracted from the image data generated by 
the image data generating section. 

[0016] According to still another aspect of the 

present invention, a program product is to be read by 
a computer of a device for controlling an imaging 
device including an optical system having an optical 
characteristic that distortion is larger in a 
peripheral area than in a central area of the image 
formed by the optical system, and an image data 
generating section for generating data of the image 
formed by the optical system. The program product 
comprising instructions of taking a picture of a 
predetermined area and waiting for appearance of an 
specified object in a stand-by mode; and tracking and 
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taking a picture of the specified object which 
appears in the predetermined area, extracting data of 
the image in the central area from the image data 
generated by the image data generating section, 
[0017] These and other objects, features and 

advantages of the present invention will become more 
apparent upon reading the following detailed 
description along with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0018] FIG. 1 is a schematic illustration of a 

monitoring system according to an embodiment of the 
present invention, 

[0019] FIG. 2 is a schematic illustration of a 

monitoring camera used in the monitoring system shown 
in Fig. 1, 

[0020] FIGS. 3A and 3B are graphs showing 

characteristics of an objective lens used in the 
monitoring earner a , 

[0021] FIG. 4 is a diagram showing an example of 

an image obtained by photographing by the monitoring 
camera , 

[0022] FIG. 5 is a block diagram of a control 

system of the monitoring camera, 

[0023] FIG. 6 is a diagram for showing a method 

for storing an original image data in an image data 
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memory , 

[0024] FIG. 7 is a table showing addresses of 

storage areas of the image data memory and original 
image data stored at these addresses, 

[0025] FIG. 8 is a diagram showing the addresses 

of the storage areas of the image data memory and 
image data of rearranged images to be stored at those 
addresses , 

[0026] FIG. 9 shows a conversion table for a red 

image , 

[0027] FIGS. 10A through 10D shows examples of 

rearranged images , 

[0028] FIG. 11 is an explanatory diagram for 

showing a moving-object detecting operation, 

[0029] FIGS. 12A and 12B are diagrams for showing 

a method for generating a conversion table, 

[0030] FIGS. 13A, 13B and 13C are diagrams for 

showing the method for generating the conversion 

table , 

[0031] FIG. 14 is a block diagram showing a 

control system of a controller, 

[0032] FIG. 15 is a flow chart showing a 

monitoring operation in a standby mode, 

[0033] FIGS. 16A and 16B are diagrams showing an 

operation of the monitoring camera when the 
monitoring camera is installed at a corner of a room 
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to be monitored, 

[0034] FIG, 17 is a flow chart showing a 

monitoring operation in a close-observation mode, and 
[0035] FIGS. 18A and 18B are diagrams showing a 

background image used in a background image 
differentiation . 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF 
THE INVENTION 

[0036] FIG. 1 is a schematic illustration of an 

arrangement of a monitoring system according to an 
embodiment of the present invention. 

[0037] As shown in FIG. 1, the monitoring system 1 

is composed of a monitoring camera 2 for capturing an 
image of a specified monitored area, a controller 3 
such as a personal computer or a cellular phone, and 
a communication network for interconnecting the 
monitoring camera 2 and the controller. 

[0038] In the monitoring system 1, when an image 

of the monitored area is captured by the monitoring 
camera 2, the obtained image data is transmitted from 
the monitoring camera 2 to the controller 3 via the 
communication network. On the other hand, when any 
request to the monitoring camera 2 is inputted in the 
controller 3, a signal representing this request 
(hereinafter, referred to as a request signal) is 
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transmitted from the controller 3 via the 
communication network to the monitoring camera 2, 
which operates in response to the request signal. 
[0039] The requests may include, for example, a 

request to establish a connection of the controller 3 
with the monitoring camera 2 and a request to switch 
the image data to be transmitted from the monitoring 
camera 2 . 

[0040] With this arrangement, the image captured 

by the monitoring camera 2 can be visually observed 
on a display section 32 (see FIG. 14) of the 
controller 3. and the operation of the monitoring 
camera 2 can be remotely controlled. 

[0041] The communication network for 

interconnecting the monitoring camera 2 and the 
controller 3 may be. for example, a radio or wireless 
LAN (local area network) built by the radio 
communication standards of Bluetooth (registered 
trademark) or using transmission medium such as radio 
waves or infrared rays or a LAN built by the 
standards of Ethernet (registered trademark). 
[0042] FIG. 2 schematically illustrates the 

monitoring camera 2 used in the monitoring system 1. 
[0043] As shown in FIG. 2, the monitoring camera 2 

is composed of a camera body 21, a substantially U- 
shaped frame 22, a geared motor 23 for changing the 
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viewing direct ion ( direct ion of monitoring) of the 
camera 21 in vertical direction (hereinafter, 
referred to as tilting direction) and a geared motor 

24 for changing the viewing direction of the camera 

21 in horizontal or right-and-left direction 
(hereinafter, referred to as panning direction). 
[0044] The camera body 21 is mounted on the U- 
shaped frame 22 with tilting direction rotational 
shafts 25 extending from the left and right surfaces 
of the camera body 21 and extending through holes 22B 
formed on side plates 22A and 22A' of the U-shaped 
frame 22. An output shaft of the geared motor 23 is 
connected to the leading end of the rotational shaft 

25 projecting through the side plate 22A. A panning 
direction rotational shaft 26 extends downward from 
the center of a bottom plate of the U-shaped frame 22, 
and an output shaft of the geared motor 24 is 
connected with the leading end of the rotational 
shaft 26. 

[0045] The geared motor 23 is fixed to the frame 

22 and so arranged to move in the panning direction 
together with the frame 22 # whereas the geared motor 
24 is fixed to a camera supporting s t rue ture ( no t 
shown ) . 

[0046] In the above arrangement, when the geared 

motor 24 is driven, the U-shaped frame 22 is rotated 
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about the rotational shaft 26, whereby the viewing 
direction of the camera 21 is changed in the panning 
direction. When the geared motor 23 is driven, the 
camera 21 is rotated about the rotatable shaft 25, 
whereby the viewing direction of the camera is 
changed in the tilting direction. 

[0047] In the following description, a motion of 

the monitoring camera 2 in which the viewing 
direction of the camera 21 is changed in the panning 
direction is referred to as a panning motion, whereas 
that of the monitoring camera in which the viewing 
direction is moved in the tilting direction is 
referred to as a tilting motion. 

[0048] In the monitoring camera 2, a wide-angle 

high-distortion lens system 201 (referred to as a 
distortion lens system hereinafter) having 

characteristics described below is adopted as an 
optical system for capturing an image of the 
monitored area. 

[0049] FIG. 3A is a graph showing a distortion vs. 

angle of view characteristic of the distortion lens 
system 201 wherein the abscissa represents distortion 
X in percent and the ordinate represents angle of 
view 6 in degree (°). FIG. 3B is a graph showing an 
angle of view vs. height of image characteristic, 
wherein horizontal axis represents angle of view 8 
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and vertical axis represents height of image Y. 
[0050] As shown in FIG. 3A, the distortion lens 

system 201 has such a characteristic that the 
distortion X takes a specified value Xi or smaller in 
a region where the angle of view 8 is small and 
suddenly increases when the angle of view 0 exceeds 
such a region . 

[0051] Here, the specified value Xi of the 

distortion X is such a value that an image with that 
value can be recognized by a person as natural and 
similar to the object without or with less distortion. 
Such an image is formed by light having passed 
through a central area of the distortion lens system 
201. For example, Xi = about 3 % (6i is about 8° at 
this time). Of course, even if the specified value 
Xi is set at a value below 3 %, e.g. about 2% or 
about 1%, the above image is recognized by a person 
as a natural image free from distortion. 

[0052] FIG. 3A shows the characteristic of the 

distortion lens system 201 having a distortion of 
about -70% at half the angle of view of about 50°. 
[0053] By this characteristic, the height 

(hereinafter, " height of image ") Y of the image 
formed by the distortion lens system 201 has a 
substantially linear relation to the angle of view 9 
in the region where the angle of view 0 is small 
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(region at the left side of dotted line in FIG, 3B) 
and has a large rate of change in relation to a unit 
change of the angle of view 8. The height of image 
here means the height of an image formed by the lens, 
of an object with a given height and located at a 
given distance from the lens e.g. at 2m. 

[0054] On the other hand, in a region where the 

angle of view 0 is large (region at the right side of 
dotted line in FIG. 3B), the height of image Y has a 
nonlinear relation to the angle of view 8, has a 
gradually decreasing rate of change in relation to 
the unit change of the angle of view 8 as the angle 
of view 8 increases and eventually takes a 
substantially constant value. 

[0055] In other words, resolution is high in the 

region where the angle of view 8 is small, whereas it 
is low in the region where the angle of view 8 is 
large . 

[0056] By suitably setting a radius of curvature 

and other optical parameter of the distortion lens 
system 201, the distortion lens system 201 has a 
wider field of view as compared to a case where a 
normal lens is used instead of the distortion lens 
system 201 to obtain the same zooming ratio as is 
obtained in a central area (corresponding to the 
"region where the angle of view 8 is small") of the 
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image where a large height of image Y can be obtained. 
At the same time, an image of an object can be formed 
in a large scale in the central area of the image as 
compared to a case where a normal lens is used 
instead of the distortion lens system 201 to obtain 
the same field of view as is obtained by the 
peripheral area (corresponding to the "region where 
the angle of view 0 is large") of the distortion lens 
system 201 . 

[0057] In this sense, the central area of the 

image formed by the distortion lens system 201 is 
referred to as a telephoto area and the peripheral 
area thereof is referred to as a wide-angle area in 
the following description, 

[0058] In this embodiment, the distortion lens 

system 201 has a focal length "f" of 80 mm for the 
telephoto area, with the focal length being measured 
as being used for a 35mm-camera and a focal length 
"f" of 16 mm for the wide-angle area, with the focal 
length being measured as being used for a 35mm-camera. 
However, the distortion lens system 201 is not 
limited thereto, 

[0059] The distortion lens system 201 has an 

optical characteristic similar to that of a human eye 
which has a highest visual power in the central 
portion of the retina called as a fovea centralis or 
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central pit, with the visual power decreasing rapidly 
towards the periphery of the retina. In other words, 
the visual power of the human eye is highest at the 
central portion of the viewing field and decreases 
rapidly as the measured portion is away from the 
central portion. The distortion lens system 201 is 
designed to form an image with the largest height of 
image at the central portion of the image and with 
the height of the image being lower in the peripheral 
portion of the image. Accordingly, this type of 
distortion lens system 201 may be called as a fovea 
lens. The fovea lens is usually composed of a 
plurality of lens components which may include 
aspherical lens. 

[0060] The fovea lens is a lens having a function 

of enlarging an image in the central area 
(corresponding to the telephoto area) of the field of 
view and compressing or contracting the image in the 
peripheral area (corresponding to the wide-angle 
area) and having a characteristic of providing 
natural images with inconspicuous distortions at a 
high resolution in the telephoto area while ensuring 
wide angle of view. 

[0061] It should be noted that the normal lens 

mentioned above is such a lens that a relationship 
between its height of image Y, focal length "f" and 
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angle of view 6 is expressed by Y = ftanG. 
[0062] When an image is captured using the 

distortion lens system 201 having the characteristic 
as described above, a captured image is such that 
objects in the peripheral area, i.e. in the wide- 
angle area is compressed by the distortion lens 
system 201. for example, as shown in FIG. 4, and an 
object or object in the central part i.e. in the 
telephoto area is enlarged as compared to the image 
of the wide angle area. 

[0063] Accordingly, the monitoring camera 2 can 

take a picture of a wide area while the central part 
of the picture is taken with a high resolution. The 
image in the telephoto area enlarged by the 
distortion lens system 201 is referred to as a 
telephoto image. 

[0064] The monitoring camera 2 of this embodiment 

has two operation modes taking advantage of the 
characteristic of the distortion lens system 201. 
[0065] Specifically, since the distortion lens 

system 201 has a wide field of view as mentioned 
above, the monitoring camera 2 is operable in a 
standby mode and a close - observation mode. In the 
standby mode, the monitoring camera 2 monitors 
whether or not there is any moving object within the 
field of view, taking advantage of the wide view of 
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the distortion lens system 201. When a moving object 
is detected in the standby mode, the camera 2 is 
switched to the close - observat ion mode wherein the 
monitoring camera 2 tracks the moving object while 
making the panning and tilting motions and takes 
picture of the moving object with a high resolution 
with the image of the tracked object being enlarged 
or magnified by the distortion lens system 201. 
[0066] FIG. 5 is a block diagram showing the 

arrangement of the control system of the monitoring 
camera 2. The monitoring camera 2 is provided with 
the distortion lens system 201, an image sensing 
section 202, a signal processor 203, an analog-to- 
digital (A/D) converter 204, an image data processor 
205 , an image data memory 206, a control unit 207, a 
driving section 208, an image data storage 209 and a 
communication interface 210, 

[0067] The lens section 201 includes an objective 

lens having the characteristic of the distortion lens 
system or fovea lens as described above for forming 
an image of an object scene to be monitored. 
[0068] The image sensing section 202 is, for 

example, a CCD color area sensor in which a plurality 
of photoelectric conversion elements such as 
photodiodes are two - dimensionally arrayed in matrix, 
color filters of R (red), G (green) and B (blue) are 
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arranged on light receiving surfaces of the 
respective photoelectric conversion elements at a 
ratio of 1: 2: 1. The image sensing section 202 
pho to - electrically converts an image of an object 
formed by the distortion lens system 201, into analog 
electrical signals (image signals) of the respective 
color components of R f G and B and outputs them as 
color image signals of R, G and B. It should be 
noted that the image sensing section 202 may be 
monochromatic instead of chromatic as mentioned above. 
[0069] The start and end of an exposure of the 

image sensing section 202 and an image sensing 
operation including the readout of the output signals 
of the respective pixels of the image sensing section 
202 (horizontal synchronization , vertical 

synchronization, signal transfer) are controlled by a 
timing generator and the like (not shown but known 
per se ) . 

[0070] The signal processor 203 applies a 

specified analog signal processing to the analog 
image signals outputted from the image sensing 
section 202, and includes a CDS (correlated double 
sampling) circuit and an AGC (auto-gain control) 
circuit, wherein the CDS circuit reduces noises in 
the image signal and the AGC circuit adjusts the 
level of the image signal. 
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[0071] The A/D converter 204 converts the analog 

image signals of R, G and B outputted from the signal 
processor 203, into digital image signals each of 
which is composed of a plurality of bits. 
[0072] The image data processor 205 applies the 

following processes to the respective digital signals 
of R, G and B converted by the A/D converter 204: a 
black level correction for correcting a black level 
to a standard black level; a white balance for 
converting the levels of the digital signals of the 
respective color components R, G and B based on a 
white standard corresponding to a light source; and a 
gamma correction for correcting gamma characteristics 
of the digital signals of the respective color 
components R, G and B. 

[0073] Hereinafter, the signal having processed by 

the image data processor 205 is referred to as an 
original image data. The pixel data of the 

respective pixels constituting the original image 
data are referred to as original pixel data. The 
image represented by the original image data is 
referred to as an original image. In this embodiment, 
the original image data of each color includes pixel 
data of 1280 x 1024 pixels. 

[0074] The image data memory 206 is a memory 

adapted to temporarily store the image data outputted 
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from the image data processor 205 and used as a work 
area for applying later- described process to this 
image data by the control unit 207. 

[0075] Here, a method for storing the original 

image data in the image data memory 206 is described. 
[0076] It is assumed that, of a storage area of 

the image data memory 206, storage areas of the 
original image data of R, G and B are virtually 
expressed in two-dimensional coordinate systems and 
the respective pixel data are arranged at grid points 
as shown in FIG. 6. It should be noted that only the 
two-dimensional coordinate systems for one color is 
shown in FIG . 6 . 

[0077] As shown in FIG. 6, the original pixel data 

of each color are successively stored in the image 
data memory 206 in a direction from the uppermost row 
to the bottommost row (direction of arrow A) and in a 
direction from left to right (direction of arrow B) 
in each row. 

[0078] Specifically, it is assumed that addrRO 

denotes an address where the pixel data of the pixel 
located at (0,0) is stored for the original pixel 
data of R, and that R(u, v), G(u, v), B(u, v) denote 
values of the pixel data of the pixels located at (u, 
v) (u = 0 to 1279, v = 0 to 1023) for three colors of 
the original image. 
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[0079] At this time, as shown in FIGS. 6 and 7, 

the original pixel data of R are successively stored 
such that R(l, 0) is stored at addr(R0+l), R(2, 0) at 

addr(R0 + 2) R( 0 , 1) at addr ( R0 + 1 2 8 0 ) , and R( 1279 , 

1023) at addr(R0+1310719) of the storage area of the 
image data memory 206. 

[0080] This can be generally expressed as follows. 

If the original image of each color is assumed to 
have M pixels along X direction and N pixels along Y 
direction, the pixel data of the pixel located at (u, 
v) is stored at addr ( RO+Mxv+u ) of the storage area of 
the image data memory 206 for the original image data 
of R. 

[0081] Further, if lt is assumed that 

addr(R0+off set) denotes an address where the pixel 
data of the pixel located at (0.0) is saved for the 
original pixel data of G. the original pixel data of 
G are successively stored such that G(l, 0) is stored 
at addr (RO+of f set+1 ) , g(2, o) at 

addr(R0+off set+2) , G(0# 1} at 

addr (RO+of fset+1280 ) , and G(1279, 1023) at 

addr(R0 + off set+1310719) of the storage area of the 
image data memory 206 as shown in FIGS. 6 and 7 
similar to the case of the image data of R. 
[0082] The general expression of this is that the 

pixel data of the pixel located at (u, v) is stored 
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at addr ( RO + of f set +Mxv + u ) of the storage area of the 
image data memory 206 for the original image data of 
G. "Offset" denotes the number of pixels constituting 
the original image of R or a larger integer and means 
that the original image data of G is saved after the 
storage area where the original image data of R is 
saved . 

[0083] Similarly, if it is assumed that 

addr ( R0 + 2xoff set ) denotes an address where the pixel 
data of the pixel located at (0,0) is stored for the 
original pixel data of B, the original pixel data of 
B are successively stored such that B(l, 0) is stored 
at addr ( R0+2xoff set +1 ) , B(2, 0) at 

addr (R0+2xof f set+2 ) , B(0 f 1) at 

addr ( R0+2xof f set+1 280 ) , and B(1279, 1023) at 

addr (R0+2xof f set+1310719 ) of the storage area of the 
image data memory 206 as shown in FIGS. 6 and 7 
similar to the case of the image data of R. 
[0084] The general expression of this is that the 

pixel data of the pixel located at (u, v) is stored 
at addr (R0+2xof f set+Mxv+u) of the storage area of the 
image data memory 206 for the original image data of 
B . 

[0085] The driving section 208 includes the geared 

motors 23 and 24 and changes the viewing direction of 
the monitoring camera 21 in panning direction and 
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tilting direction in response to a command from the 
control unit 207. 

[0086] The image data storage 209 includes a hard 

disk or the like and adapted to save an image file 
generated by a later - described saved image generator 
2077 of the control unit 207. 

[0087] The communication interface 210 is an 

interface based on the standards of the radio or 
wireless LAN, Bluetooth ( regis tered trademark), 
Ethernet (registered trademark) and the like, and 
adapted to transmit the image data to the controller 
3 and receive the request signal from the controller 
3 . 

[0088] The control unit 207 is composed of a 

microcomputer having a built-in storage (storage 2079 
to be described later) including, for example, a ROM 
for storing a control program and a RAM for 
temporarily storing data. The control unit 207 

organically controls the driving of the respective 
members provided in the aforementioned camera body 21 
and the camera system to generally control the image 
capturing operation of the monitoring camera 2. 
[0089] The control unit 207 is provided, as 

functional blocks or units, with an image rearranging 
unit 2071, a mo ving - ob j ec t detector 2072, a mode 
switch controller 2073, a power supply controller 
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2074, a sensing controller 2075, a drive controller 
2076, the saved image generator 2077, a communication 
controller 2078 and the storage 2079. 

[0090] Here, since the original image is generally 

distorted as described above, a distorted image is 
displayed on the display section 32 of the controller 
3 if a data of the original image is transmitted to 
the controller 3 as it is. In that case, a 

satisfactory image visibility (natural looking of the 
image like as the scene and object are viewed by 
human eyes) cannot be obtained. 

[0091] Further, since the original image data 

contains a relatively large amount of information or 
a volume of data, a communication of the image data 
between the monitoring camera 2 and the controller 3 
takes much time and, therefore, the image display on 
the display section 32 of the controller 3 may not be 
synchronized with the image sensing operation of the 
image sensing section 202 performed, for example, 
every 1/30 seconds. 

[0092] In order to solve such a problem, the image 

rearranging unit 2071 performs a process to correct 
the distortion created by capturing an image of the 
object using the distortion lens system 201 and 
generate a rearranged image whose number of pixels is 
smaller than that of the original image (hereinafter 
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referred to as rearranging process). In this 

embodiment, the number of pixels of the rearranged 
image is 640 x 480. 

[0093] The rearranging unit 2071 selects a 

suitable conversion table T corresponding to the 
operation mode (standby mode or close - observation 
mode) of the monitoring camera 2 from a plurality of 
later - described conversion tables T stored in the 
storage 2079 beforehand; extracts a part of the 
pixels of the original images using the selected 
conversion table T ; arranges the extracted pixels to 
generate an image (rearranged image) by the extracted 
pixels; and stores image data of this rearranged 
image in a storage area of the image data memory 206 
which is different from the area where the image data 
of the original image is saved. 

[0094] In the rearranging process, the pixels 

extracted from the original image in the standby mode 
are the pixels of a partial or entire image of the 
wide-angle area and the image of the telephoto area. 
On the other hand, the pixels extracted from the 
original image in the clo s e - observa t ion mode are the 
pixels of the telephoto area, generating an image 
shown in FIG. 10B as will be described later. 
[0095] Here, upon describing the rearranging 

process, the respective storage areas for image data 
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of R, G and B of the rearranged image in the storage 
areas of the image data memory 206 are virtually 
expressed in two-dimensional coordinate systems like 
as in the case shown in FIG. 6, and the rearranged 
image is generated by arranging the extracted pixels 
at the grid points of other two-dimensional 
coordinate systems . 

[0096] In order to distinguish the two-dimensional 

coordinate systems set for the rearranging process 
from those set for storing the original image, the 
former two-dimensional coordinate systems are 
referred to as rearrangement coordinate systems. It 
should be noted that only the two-dimensional 
coordinate systems for one color is shown in FIG. 8. 
[0097] FIG. 9 shows the conversion table T for the 

image of R. As shown in FIG. 9, the conversion table 
T shows correspondence between the addresses of the 
pixel data of the original image data stored in the 
image data memory 206 and the respective coordinates 
(I, J) (1 = 0 to 639 , J = 0 to 479 ) of the 
rearrangement coordinate systems where the designated 
pixels are arranged or located. 

[0098] In the conversion table T shown in FIG. 9, 

addrR(i, j) denotes an address where the original 
pixel data of R to be arranged at (i, j) in the 
rearrangement coordinate systems of R is stored. For 
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example, of the pixels of the original pixel data of 
R, the pixel corresponding to the pixel data stored 
at addrR(0, 0) of the storage area of the image data 
memory 206 is arranged at (0, 0) in the rearrangement 
coordinate systems of R. 

[0099] It is described above that, if the original 

image is assumed to have M pixels in X-direction and 
N pixels in Y-direction, the pixel data of the pixel 
located at (u, v) is saved, for example, at 
addr (RO + Mxv + u ) in the original image data of R. If 
the pixel located at (u, v) in the two-dimensional 
coordinate systems set for the original image is 
assumed to be arranged at (i, j ) in the rearrangement 
coordinate systems, addrR(i, j) corresponds to 
addr ( RO+Mxv+u ) . 

[0100] Similarly, if addrG(i, j) is assumed to 

denote an address where the original pixel data of G 
to be arranged at (i, j) in the rearrangement 
coordinate systems of G is stored. Of the pixels of 
the original pixel data of G, the pixel corresponding 
to the pixel data stored at addrG(i, j ) , i.e. 
addr ( RO+of f set+Mxv+u ) of the storage area of the 
image data memory 206 is arranged at (i, j ) in the 
rearrangement coordinate systems of G. 

[0101] Similarly, if addrB(i, j) is assumed to 

denote an address where the original pixel data of B 
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to be arranged at (i, j) in the rearrangement 
coordinate systems of B is stored. Of the pixels of 
the original pixel data of B, the pixel corresponding 
to the pixel data saved at addrB(i, j), i.e. 
addr ( R0+2xof f set+Mxv+u ) of the storage area of the 
image data memory 206 is arranged at (i, j) in the 
rearrangement coordinate systems of B. 

[0102] In this way, the conversion table T of this 

embodiment defines a method for extracting a part of 
the pixels from 1280 x 1024 pixels of the original 
image and arranging them at 640 x 480 grid points in 
the rearrangement coordinate systems for each color. 
[0103] Accordingly, the rearranged image is 

generated in which the distortion created in the 
original image is corrected and the pixel number is 
reduced as compared to the original image, for 
example, rearranged image as shown in FIG. 10A is 
generated from the original image shown in FIG. 4. 
[0104] As described above, a plurality of 

different conversion tables are prepared beforehand 
in this embodiment, and the image rearranging unit 
2071 performs the rearranging process by selecting 
the conversion table T in accordance with the 
selected operation mode (standby mode or close- 
observation mode) of the monitoring camera 2, or in 
response to a command from the controller 3, or in 
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accordance with other condition. 

[0105] For example, in the standby mode, the image 

rearranging unit 2071 selects a conversion table Tl 
for generating a rearranged image from the data of 
the entire original image, the rearranged image 
showing a wide area, and generates a rearranged image 
showing a relatively wide area, for example, as shown 
in FIG. 10A, using this conversion table Tl. 
Hereinafter, this rearranged image in the standby 
mode is referred to as a wide-angle image. 
[0106] On the other hand, in the close - observation 

mode, the image rearranging unit 2071 extracts the 
image of the central area (image captured in the 
telephoto area) from the original image, selects a 
conversion table T2 for generating a rearranged image 
showing the image of the central area including a 
moving object, generates from the extracted pixel 
data such a rearranged image in which the moving 
object is enlarged as compared to the rearranged 
image generated in the standby mode as shown in FIG. 
10B, using the conversion table T2 . Hereinafter, 
this rearranged image in the close - observation mode 
is referred to as a close - observation image. 
[0107] In this way, when the rearranged images 

generated in the respective operation modes are 
transmitted to the controller 3, an operator of the 
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controller 3 can observe a wide area on the display 
section 32 in the standby position, whereas he or she 
can exactly and certainly observe the features of the 
moving object in the close - observation mode. 
[0108] For the close - observation mode, other 

conversion tables T3 and T4 are also provided for 
showing two kinds of images at a same time as shown 
in FIGS. 10C and 10D. 

[0109] The image rearranging unit 2071 generates 

one rearranged image in which reduced images of the 
respective rearranged images shown in FIG. 10A and 
10B are arranged one above the other with a specified 
interval therebetween as shown in FIG. 10C, using the 
conversion table T3, when the controller 3 designates 
the conversion table T3 as described later. 
[0110] When it is judged that the moving object 

cannot be captured in the telephoto area by the 
monitoring camera 2, the image rearranging unit 2071 
selects the conversion table T4 and generates one 
rearranged image in which a reduced image showing a 
more extended area including the moving object than 
in the image at the upper part of FIG. 10C and a part 
of the rearranged image shown in FIG. 10A are 
arranged one above the other with an interval 
therebetween. 

[0111] In this way, the wide-angle image shown in 
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FIG. 10A and the close - observation image shown in FIG. 
10B are selectively displayed on the display section 
32 (see FIG. 14) of the controller 3. Thus, 
switching operation for the display by means of an 
operating or manipulation section 31 is required for 
visually recognizing the two images. However, by 
simultaneously displaying two kinds of images as 
shown in FIGS. IOC and 10D, more secure monitoring 
can be conducted without requiring the user of the 
controller 3 to switch the display between the wide- 
angle image display and the close-observation image 
display by means of the manipulation section 31. 
[0112] It should be noted that "SE" (south east), 

"E" (east), "NE" (north east) shown in FIGS. 10C and 
10D denote directions in which the monitoring camera 
2 views . 

[0113] The moving-object detector 2072 detects an 

moving object in an original image by a time 
differentiation process described below. 

[0114] The time differentiation is a process of 

determining differences between or among a plurality 
of images photographed at specified relatively short 
intervals and detecting an area having a change 
between or among the images (changed area). 
[0115] As shown in FIG. 11, the moving - obj ect 

detector 2072 extracts a changed area using three 
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images: a present image 510, an image 511 
photographed a little earlier than the present image 
510, and an image 512 photographed a little earlier 
than the image 511. 

[0116] The image 510 includes an area 513 where a 

moving object is expressed. However, the area 513 
expressing the moving object cannot be extracted from 
the image 510 only. 

[0117] The image 511 includes an area 514 where 

the moving object is expressed. Although the same 
moving object is expressed in the areas 513 and 514, 
the positions thereof in the images 510 and 511 
differ from each other since the images 510 and 511 
of the moving object are photographed at 1 different 
points of time. 

[0118] A differentiated image 520 is obtained by 

differentiating the images 510 and 511. The 
differentiated image 520 includes the areas 513 and 
514, with the image commonly existing in the images 
510 and 511 being removed in the image 520 by the 
differentiation. The area 513 in the differentiated 
image 520 is an area expressing the moving object 
which was present at a position at the time when the 
image 510 was photographed. The area 514 in the 
differentiated image 520 is an area expressing the 
moving object which was present at a position at the 
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time when the image 511 was photographed. A 
differentiated image 521 is obtained by 

differentiating the images 510 and 512, and includes 
the area 513 and an area 515, with the image commonly 
existing in the images 511 and 512 being removed in 
the image 521 by the differentiation. The area 513 
in the differentiated image 521 is an area expressing 
the moving object, and is present at a position at 
the time when the image 510 was photographed. The 
area 515 in the differentiated image 521 is an area 
expressing the moving object which was present at a 
position at the time when the image 512 was captured. 
[0119] Next, an image 530 is obtained by taking a 

logical multiplication of the differentiated images 
520 and 521. As a result, the image 530 includes 
only the area 513 expressing the moving body at the 
time when the image 510 was captured. Thus. the 
moving body and its position is detected. 
[0120] The mode switch controller 2073 switches 

the operation mode between the standby mode in which 
the monitoring camera 2 is fixed in a predetermined 
posture (initial posture) to capture an image of the 
entire monitored area and the close - observat ion mode 
in which the monitoring camera 2 is caused to track 
the moving object displaying the image of the 
telephoto area. 
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[0121] The mode switch controller 2073 switches 

the operation mode to the close-observation mode to 
monitor the features of the moving object in detail 
when the moving object is detected in the standby 
mode. The operation mode is switched to the standby 
mode to widely monitor the monitored area when the 
following close - observation mode ending conditions 
are satisfied in the close-observation mode. 
[0122] In this embodiment, three close - observation 

ending conditions are provided for switching the 
operation mode from the close - observation mode to the 
standby mode: 

(1) The moving object has moved out of the field 
of view, 

(2) A specified period has passed after the 
moving object stopped within the view, and 

(3) A specified period has passed after the 
operation mode was switched to the clo s e - observa t ion 
mode . 

[0123] When any of the above conditions is 

satisfied, the operation mode is switched from the 
close-observation mode to the standby mode. 
[0124] The c lo s e - ob s erva t ion ending conditions 

include the condition that the moving body has moved 
out of the field of view (condition (1)), when the 
moving object is thought to have left the monitored 
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area . 

[0125] The close-observation ending conditions 

include the condition that the specified period has 
passed after the moving object stopped within the 
field of view (condition (2)). in this condition, it 
is expected that the closely observed object will 
stop for a relatively long time as the motion of the 
closely observed object keep stopping for the 
specified period and other moving object(s) may be 
overlooked if such a closely observed object is 
persistently observed in the clos e - observa t ion mode 
having a narrower field of view. 

[0126] The close-observation ending conditions 

include the condition that the specified period has 
passed after the operation mode was switched to the 
close-observation mode (condition (3)) because other 
moving object(s) may be overlooked if such a closely 
observed object is persistently observed for a long 
time in the close - obs ervat ion mode having a narrower 
view similar to the case of the condition (2), and 
the storage capacity of the image data storage 209 
can be effectively used. 

[0127] Upon receiving a request to establish a 

communication connection from the controller 3, the 
mode switch controller 2073 establishes the 
connection and then sets a r emo t e - con t rol mode for 
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receiving various requests such as a request to 
change the posture of the monitoring camera 2. This 
remote - control mode is canceled if no request is made 
during a specified period. 

[0128] The power supply controller 2074 controls 

on-off of the power supply of the monitoring camera 2 
when a power switch (not shown) provided on the 
monitoring camera 2 is operated, and restricts a 
preliminary power supply to the driving section 208 
such as the geared motors 23 and 24 and the 
communication interface 210 in the standby mode for 
energy saving. 

[0129] The sensing controller 2075 causes the 

image sensing section 202 to sense images, for 
example, at intervals of 1/30 seconds in the standby 
mode while causing the image sensing section 202 to 
sense images at intervals shorter in close- 
observation mode than the intervals for the standby 
mode . 

[0130] A time interval between the image sensing 

operations of the image sensing section 202 in the 
close-observation mode is set shorter than the one in 
the standby mode in order to carefully monitor the 
movement of the moving object. By setting the time 
interval between the image sensing operations of the 
image sensing section 202 in the standby mode 
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relatively long, it can be prevented or suppressed 
that the close-observation image having a higher 
importance than the wide-angle image cannot be saved 
in the image data storage 209, 

[0131] The drive controller 2076 controls the 

rotations of the geared motors 23 and 24 of the 
driving section 208. The drive controller 2076 stops 
the rotations of the geared motors 23 and 24 of the 
driving section 208 and fixes the monitoring camera 2 
in the initial posture in the standby mode, whereas 
it drives the geared motors 23 and 24 to cause the 
monitoring camera 2 to track the moving object in the 
close -observation mode . 

[0132] The saved image generator 2077 generates a 

compressed image data by applying a specified 
compression by the MPEG (moving picture experts 
group) method to the pixel data of the rearranged 
image, and saves an image file obtained by adding 
data of the photographed image (including metha data 
and compression rate) to the compressed image data, 
[0133] In this embodiment, two kinds of 

compression rates are provided corresponding to the 
operation modes (standby mode and close - observat ion 
mode) of the monitoring camera 2, and the image is 
compressed at a relatively small compression rate in 
the close - observat ion mode so as to obtain 
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information of detailed features of the moving object. 
[0134] On the other hand, in the standby mode, the 

image is not required to have a high resolution so 
long as the moving object is detectable, and the 
image is compressed at a compression rate larger than 
the one used in the close - observation mode in order 
to save storage area of the image data storage 209 
for the close - observation image having a higher 
importance than the wide-angle image. 

[0135] The metha data is generally a data bearing 

information for identifying a subject data (e.g. data 
of the image captured by the monitoring camera 2 in 
this embodiment) which information is referred to 
retrieve the subject data from a multitude of data. 
A desired image can be easily retrieved from a 
plurality of images stored in the image data storage 
209 by adding this metha data to the image data. 
[0136] The communication controller 2078 

establishes and breaks a communication connection of 
the monitoring camera 2 with the controller 3, and 
controls the transfer of the image data and the like 
from the image data memory 206 to the communication 
interface 2 10. 

[0137] The storage 2079 includes a plurality of 

conversion tables T used by the image rearranging 
unit 2071 to generate rearranged images as described 
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above. The conversion tables T are designed to 
determine beforehand how the pixel data of the pixels 
extracted from the original image are to be arranged 
in order to correct the distortion of the original 
image and to change the number of pixels and the size 
of the photographing area. 

[0138] Hereinafter, a method for generating the 

conversion table T is described. 

[0139] Assuming that the number of pixels of the 

original image is M x N as shown in FIG. 12A and the 
number of pixels of the rearranged image is K x L, 
coordinates (u, v) of a pixel Q of the original image 
corresponding to an arbitrarily selected pixel 
(hereinafter referred to as a referred pixel B, 
coordinates B(i, j ) ) of the rearranged image are 
calculated . 

[0140] First, as shown in FIG. 12B, a distance d 

(dx: x-component, dy: y-component) from a center A 
(K/2, L/2) of the rearranged image to the referred 
pixel B is: 

dx = (K/2- j ) ... ( 1 ) 

dy = (L/2- j } ... (2) 

d = ^{dx 2 +rfy 2 ) ... ( 3 ) 

d = ^{(K/2-i) 2 +(L/2-j) 2 } ... (4) 
[0141] If it is assumed that the rearranged image 

shown in FIG. 12B is photographed using a normal lens 
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whose relationship among the height of image Y, the 
focal length f and the angle of view 0 is expressed 
by Y = f-tan 9, an angle of incidence ()> of the light 
converted into the pixel data of the pixel located at 
the coordinate (i, j) in the rearranged image is the 
same as the angle of incidence of the light of the 
pixel data of the pixel located at the coordinate (u, 
v) in the original image. 

[0142] Accordingly, if an angle of view of the 

rearranged image in horizontal plane is a radian, the 
angle of incidence of the light converted into the 
pixel data of the pixel located at the coordinate (i, 
j) in the rearranged image can be expressed as 
follows . 

[0143] First, following two equations hold as can 

be seen from FIGS, 13A to 13C: 

f = (K/2)/tan(a/2) ... (5) 

tan<|>= d/f ( 6 ) 

Thus , 

<|> = tan _1 {d/ (k/2 ) /tan(a/2 ) } ... (7) 

[0144] If h denotes a distance (height of image) 

between a center P (M/2, N/2) and the coordinates Q(u, 
v) in the original image, the distance h is expressed 
as a function of the angle of incidence ((> calculated 
by equation (7). 

h = f (<|>) ... (8) 
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This function is determined according to a radius of 
curvature and other optical parameters of the 
distortion lens system 201. 

[0145] On the other hand, following two equations 

hold as can be seen from FIGS. 12A and 12B. 

h : d = (u-M/2 ) : dx ( 9 ) 

h : d - ( v-N/2 ) : dy ... ( 10 ) . 

[0146] From equations (9), (10), following 

equations (11), (12) are obtained. 

u = M/2 + h x (dx/d) ... (11) 

v = N/2 + h x (dy/d) ... (12) 

[0147] In accordance with equations (8), (11) and 

(12), the coordinates (u, v) of the pixel data in the 
original image corresponding to the pixel data 
located at the coordinates (i, j) can be obtained. 
[0148] The pixel data of the pixel located at the 

thus obtained coordinates (u, v) in the original 
image is stored at addr ( R0+Mx v + u ) of the image data 
memory 206. When the rearranged image is generated 
by the image rearranging unit 2071 using the 
conversion table T (see FIG. 9), the pixel data at 
this address addr ( R0 +Mx v+u ) is arranged at addr(i, j) 
stored in the conversion table T. 

[0149] On the other hand, the controller 3 

includes the manipulation section 31, the display 
section 32, a controlling section 33 and a 
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communication interface 34 as shown in FIG. 14. 
[0150] The manipulation section 31 is adapted for 

inputting commands (hereinafter, "instruction 

commands") to give the monitoring camera 2 various 
instructions such as making it to perform the panning 
and tilting motions, and the storing and transmission 
of the image data. The manipulation section 31 may 
take a form of a keyboard and a mouse in the case 
where the controller 3 is a personal computer 
(hereinafter, "PC"), whereas it may take a form of a 
set of push buttons in the case where the controller 
3 is a cellular phone. 

[0151] The display section 32 is adapted for 

displaying images due to the image data transmitted 
from the monitoring camera 2 via the communication 
network, and may take a form of a monitor in the case 
where the controller 3 is a PC while it may take a 
form of, for example, to a liquid crystal display in 
the case where the controller 3 is a cellular phone. 
[0152] The controlling section 33 includes a 

microcomputer having built-in ROM 121 for storing, 
for example, a control program and RAM 122 for 
temporarily storing the data, and generally controls 
operation of the controller 3 by organically 
controlling the manipulation section 31, the display 
section 32, the communication interface 34, etc. 
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[0153] The controlling section 33 includes a 

command generator 331 which, upon the input of a 
specified instruction to the monitoring camera 2 from 
the manipulation section 31, generates an instruction 
command corresponding to the inputted instruction and 
sends the instruction command to the communication 
interface 34 . 

[0154] The instruction commands includes a command 

to request a communication process to establish a 
communication connection between the controller 3 and 
the monitoring camera 2, a command to instruct the 
panning motion and the tilting motion of the 
monitoring camera 2, a command to request the 
transmission of the image data stored in the image 
data storage 209 of the monitoring camera 2, a 
command to request a switching of image data to be 
transmitted in order to switch the image display mode 
on the display section 32, for example, between the 
one shown in FIG. 10B and the one shown in FIG. 10C, 
and a command to request the communication process to 
break the communication connection of the controller 
with the monitoring camera 2 . 

[0155] The communication interface 34 is an 

interface based on the standards of the radio LAN, 
Bluetooth (registered trademark), Ethernet 

(registered trademark), and the like, and adapted to 
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receive the image data from the monitoring camera 2 
and transmit the instruction commands to the 
monitoring camera 2 . 

[0156] Next, the monitoring operations by the 

monitoring camera 2 according to this embodiment are 
described. It should be noted that a remote control 
of the monitoring camera 2 from the controller 3 is 
assumed to be accepted only in the standby mode in 
order to simplify the following description. 
[0157] FIG. 15 is a flow chart showing a series of 

monitoring operations carried out in the standby mode, 
and FIG. 16 is a diagram showing the operation of the 
monitoring camera 2 in the case that the monitoring 
camera 2 is installed at a corner of a room to be 
monitored . 

[0158] As shown in FIG. 15, the geared motors 23 

and 24 are first controlled by the drive controller 
2076 in the standby mode and the monitoring camera 2 
is set in its initial posture where the entire area 
to be monitored is monitored as shown in FIG. 16A 
( Step #1 ) . 

[0159] Thereafter, a power-saving mode is set by 

the power supply controller 2074 in order to save 
energy, whereby power supply to the geared motors 23 
and 24 and other components to be at rest is 
restricted (Step #2). Then, the detection of a 
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moving object is started by the moving- obj ect 
detector 2072 while an image data of an image 
photographed by the image sensing operation of the 
image sensing section 202 is stored in the image data 
storage 209 (Step #3). 

[0160] A rearranged image showing a wide area, for 

example, as shown in FIG. 10A is generated using the 
conversion table T (Step #4) and stored in the image 
data storage 209 (Step #5). 

[0161] When a signal for requesting the 

communication connection is received from the 
controller 3 via the communication interface 210 (YES 
at Step #7) before any moving object is detected (NO 
at Step #6), the communication connection of the 
monitoring camera 2 with the controller 3 is 
established by the communication controller 2078 
(Step #8 ) . 

[0162] At this stage, the communication controller 

2078 generates a reception signal representing that 
signal for requesting the communication connection 
has been received from the controller 3 and the 
communication interface 210 transmits the reception 
signal to the controller 3, thereby establishing the 
communication connection between the monitoring 
camera 2 and the controller 3. 

[0163] Upon the establishment of the communication 
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connection between the monitoring camera 2 and the 
controller 3, the remot e - control mode for receiving 
the requests from the controller 3 is set after the 
power-saving mode is canceled by the power supply 
controller 2074 (Step #9). 

[0164] In the r emote - control mode, upon being 

requested from the controller 3 to perform, for 
example, the panning motion, the tilting motion, the 
transmission of the image data (YES at Step #10), the 
monitoring camera 2 operates in response to this 
request (Step #11). 

[0165] Specifically, when the communication 

interface 210 receives a pan/tilt command, the 
panning motion and the tilting motion are conducted 
by the drive controller 2076 in response to this 
command. When a stored image transmission command is 
received, the image data stored in the image data 
storage 209 is transmitted by the communication 
controller 2078 and the communication interface 210 
in response to this command. 

[0166] When an image switching command is received, 
the image rearranging unit 2071 switches, in response 
to this command, the conversion table T to be used. 
When a connection end command is received, the 
communication connection between the monitoring 
camera 2 and the controller 3 is broken or cut off by 
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the communication controller 2078 in response to this 
command. In this embodiment, since the conversion 
tables used to generate the rearranged images can be 
switched from one to another in the close - observation 
mode as described above, switching of the conversion 
table is made among or between the conversion tables 
for generating the rearranged image in the close- 
observation mode. 

[0167] The process returns to Step #2 if no 

request has been made from the controller 3 even 
after the lapse of a specified period following the 
setting of the remote - control mode (NO at Step #10 
and YES at Step #12). 

[0168] The process returns to Step #6 unless the 

communication interface 210 receives the 

communication connection requesting signal from the 
controller 3 (NO at Step #7) before the moving object 
is detected (NO at Step #6). 

[0169] When the moving object is detected by the 

moving-object detector 2072 (YES at Step #6), the 
operation mode of the monitoring camera 2 is switched 
to the close - observat ion mode by the mode switch 
controller 2073 (Step #14) after the power-saving 
mode is canceled by the power supply controller 2074 
( Step #13 ) . 

[0170] FIG. 17 is a flow chart showing a series of 
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monitoring operations in the clos e - ob s ervat ion mode. 
[0171] In the clo se - obs ervat ion mode, the 

detection of a moving object by the moving-object 
detector 2072 is started while the image data of the 
image captured by the image sensing operation of the 
image sensing section 202 is being stored in the 
image data storage 209 (Step #20). 

[0172] When a moving object is detected by the 

moving-object detector 2072 (YES at Step #21), the 
drive controller 2076 starts the operation control of 
the geared motors 23 and 24, i.e. the panning motion 
and the tilting motion of the monitoring camera 2 
( Step #22 ) . 

[0173] For example, when a moving object appears 

in the monitored scene and moves as shown by an arrow 
P in FIG. 16B, the monitoring camera 2 is driven to 
change its viewing direction in a direction of an 
arrow Q from the initial viewing direction shown in 
FIG. 16A. 

[0174] If the moving object is captured in the 

telephoto area by panning and tilting the monitoring 
camera 2 (YES at Step #23), a rearranged image 
showing the moving object in relatively large scale, 
for example, as shown in FIGS. 10B or 10C is 
generated using the conversion table T2 or T3 (Step 
#24) . 
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[0175] On the other hand, if the movement of the 

moving object is not tracked by the panning and 
tilting motions of the monitoring camera 2 and the 
moving object is not captured in the telephoto area 
(NO at Step #23), a rearranged image showing a more 
extended area including the moving object as shown in 
FIG. 10D as compared to the images shown in FIGS. 10B 
and 10C is generated by using the table T4 (Step #25). 
[0176] The data of the rearranged image generated 

at Step #24 or #25 is stored in the image data 
storage 209 by the saved image generator 2077 and 
this image data is transmitted to the controller 3 
via the communication controller 210 by the 
communication controller 2078 (Step #26). 

[0177] Thereafter, the mode switch controller 2073 

determines whether or not the close - observation mode 
ending condition is satisfied such as the exit of the 
moving object from the close - observat ion area or the 
lapse of the specified period after the operation 
mode of the monitoring camera 2 was switched to the 
close - observation mode (Step #27). The operations in 
Steps #20 through #26 are repeated while the close- 
observation mode ending condition is not satisfied 
(NO at Step #27 ) . 

[0178] On the other hand, if the close - observation 

ending condition is satisfied (YES at Step #27), the 
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operation mode is switched to the standby mode by the 
mode switch controller 2073 (Step #29) after the 
monitoring camera 2 is reset to the initial posture 
by the drive controller 2076 and the driving section 
208 ( Step #28 ) . 

[0179] In this way, in the close-observation mode, 

the distortion lens system 201 is turned, with the 
image of the moving object being formed on the image 
sensing section 202 by the distortion lens system 201, 
and the rearranging process is carried out by 
extracting the pixels of the telephoto area portion 
of the original image which has a high resolution. 
Thus, the c lo s e - ob servat ion image showing the moving 
object in relatively large scale with no or little 
distortion can be obtained. As a result, an image 
with satisfactory visibility is displayed on the 
display section 32 of the controller 3. 

[0180] In the standby mode as well, the 

rearranging process is carried out by extracting the 
pixels of the image in the telephoto area and a part 
or all of the peripheral or wide-angle area of the 
original image. Thus, a rearranged image showing a 
wider area as compared to the clo s e - obs ervat ion image 
and having no or little distortion can be obtained. 
As a result, the monitored area can be monitored by 
the controller 3 also in the standby mode. 
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[0181] Further. as a plurality of conversion 

patterns are provided with for the rearranging 
process, various close - observation images and various 
wide-angle images having different pixel numbers, 
different image display areas or different sizes of 
the image of the moving object can be obtained. 
[0182] When the panning motion and the tilting 

motion of the monitoring camera 2 fails to follow the 
moving object in the close-observation mode. the 
monitoring camera 2 is switched to show the image of 
the wide-angle area in which the image of the moving 
object is expected to be included. The wide-angle 
area image is displayed singly as shown in FIG. 10A 
or along with the image of the telephoto area as 
shown in FIG. 10D. Thus, the features of the moving 
object can be monitored on the display section 32 of 
the controller 3 even if the moving object cannot be 
tracked by the panning motion and the tilting motion 
of the monitoring camera 2 . 

[0183] Further, since the operation mode is 

switched to the close - observation mode when the 
moving object is detected in the standby mode while 
the mode is switched to the standby mode when the 
close-observation mode ending condition is satisfied 
in the close-observation mode, the monitored area can 
be widely displayed in the standby mode until the 
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moving object appears and the moving object can be 
monitored in detail in the close-observation mode 
when the moving object appears* 

[0184] Since the metha data representing that the 

image is the close - observation image is attached to 
the image data of the close - observation image upon 
storing the image data of the close - observation image 
in the image data storage 209 , a desired close- 
observation image can be easily retrieved from a 
plurality of close-observation images stored in the 
image data storage 209. 

[0185] Further, since the image sensing section 

202 is caused to perform the image sensing operations 
at shorter intervals in the close - observation mode 
than in the standby mode, more data of the moving 
object can be obtained in detail in the close- 
observation mode and it can be prevented or 
suppressed that the data of close - observation images 
having a higher importance than the data of the wide- 
angle images is not stored. 

[0186] Furthermore, since the close - observation 

images are compressed at a lower compression ratio 
than the wide-angle images, the data of the moving 
object can be obtained in more detail in the close- 
observation mode than in the standby mode, and it can 
be prevented or suppressed that the data of close- 
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observation images having a higher importance than 
the data of the wide-angle images is not stored. 
[0187] Further, the controller 3 is provided with 

the command generator 331 for generating the 
instruction command to instruct the switching of the 
conversion tables, and the conversion table is 
switched in the monitoring camera 2 when the 
instruction command is transmitted from the 
controller 3 to the monitoring camera 2 via the 
communication interfaces 210 and 34 of the monitoring 
camera 2 and the controller 3. Thus, the conversion 
tables or the displayed images can be remotely 
switched by the controller 3. 

[0188] Further, since the close-observation image 

and the wide-angle image are generated only by 
rearranging the pixels of the original image, it can 
be avoided to complicate the construction of the 
control unit 207. 

[0189] The present invention is not limited to the 

foregoing embodiment but may be modified or varied in 
variously ways such as, by for example, described in 
the followings (1) through (13). 

[0190] (l) The process for detecting the moving 
object is not limited to the aforementioned time 
differentiation. For example. a background image 
differentiation may be adopted in which a background 



54 



area to be monitored may be specified beforehand, and 
an area not found in the background image is detected 
as a changing area based on a difference between a 
background image obtained by capturing an image of 
the background area beforehand and an image obtained 
by capturing an image of the present background area. 
[0191] FIGS. 18A and 18B are diagrams for 

explaining a background image used in the background 
image differentiation, wherein FIG. 18A shows a 
background area and a presence permitted area, and 
FIG. 18B shows a relationship between the background 
area and an image capturing capable range of the 
camera . 

[0192] As shown in FIG. 18A, a background area 601 

is a range which can be monitored at a time by the 
camera 21 and includes a presence permitted area 602 
which is an area specified beforehand in relation to 
the background image 601. 

[0193] As shown in FIG. 18B, a plurality of 

background areas (rectangular areas delineated by 
solid lines) are arranged within the image capturing 
capable range 600 of the camera 21 such that 
adjoining background areas partly overlap each other. 
The presence permitted areas (the rectangular area 
delineated by broken lines) included in the 
background areas adjoin each other without 
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overlapping with the presence permitted areas of 
adjoining background areas. For example, the 

background areas 601A and 601B overlap each other at 
the hatched portions, but the presence permitted 
areas 602A and 602B adjoin each other without 
overlapping each other. 

[0194] By arranging the background areas and the 

presence permitted areas as mentioned above, a moving 
object within the image capturing capable range of 
the camera is present in any one of the presence 
permitted areas except a part of a peripheral area of 
the photographing capable range. Accordingly, the 
changing area can be tracked without any 
consideration of the moving direction or moving speed 
of the changing area or without predicting the 
position to which the changing area will move if the 
image capturing range of the camera is switched to 
the background area including the presence permitted 
area where the changing area is present. 

[0195] Since the capturing capable range of the 

camera is divided into a plurality of sections to 
arrange a plurality of background areas with less 
overlapping, the capacity of saving the background 
image obtained by monitoring the background area can 
be reduce. 

[0196] (2) Other processes may be adopted for 
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detecting the moving object. For example, a color 
detection may be adopted in which a specific color, 
for example, a color of human skin is detected from 
an image and extracted therefrom. 

[0197] (3) Although the rearranged image data is 
stored in the image data storage 209 built in the 
monitoring camera 2 in the aforementioned embodiment, 
the present invention is not limited thereto. For 
example, a computer (or a server) may store the 
rearranged image data in a case where the monitoring 
camera 2 is connected via a communication network 
with the computer for performing process, such as 
storage and provision of image data, in response to 
requests from a specified client unit including the 
controller 3 . 

[0198] (4) Although the image having 640 x 480 

pixels is generated from the original image having 
1280 x 1024 pixels by the rearranging process in the 
foregoing embodiment, the present invention is not 
limited thereto. For example, an image having 320 x 
240 pixels may be generated. 

[0199] (5) In the case where the monitoring 

system 1 includes a plurality of monitoring cameras 2 , 
specific IDs (identifications) may be given to the 
respective monitoring cameras 2 and the IDs of the 
monitoring cameras 2 as communication partners are 
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registered in the controller 3. When any of the 
monitoring cameras 2 is remotely controlled by the 
controller 3, various data including image data are 
transmitted and received after the ID of the selected 
monitoring camera 2 is designated by means of the 
manipulation section 31 of the controller 3 and a 
communication connection is established between this 
monitoring camera 2 and the controller 3. 
[0200] (6) If the controller 3 is provided with a 

notifying device such as a light emitting device or a 
sound generator, the detection of the moving body may 
be notified to a user of the controller 3 by means of 
this notifying device. 

[0201] (7) Although the image data is stored in 

the image data storage 209 not only in the close- 
observation mode but in the standby mode in the 
foregoing embodiment, the present invention is not 
limited thereto. The data of the image photographed 
in the standby mode may not be stored in the image 
data storage 209. 

[0202] (8) An external sensor 40 for detecting, 

for example, that a window pane was broken, may be 
provided with to communicate with the monitoring 
camera 2 as shown in FIG. 5. In that case, the 
monitoring camera 2 may start monitoring upon the 
receipt of a detection signal from the external 
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sensor 4 0 . 

[0203] In more detail, the monitoring camera 2 may 

be provided with a signal input/output device 50 to 
receive the detection signal from the external sensor 
40 and output a switch control signal to turn on and 
off the power supply to the external sensor 4 by 
means of this signal input/output device 50. If an 
external equipment other than the external sensor 40 
is connected with the monitoring camera 2 for 
communication, various signals including the above 
switch control signal may be transmitted and received 
between the external equipment and the monitoring 
camera 2 . 

[0204] (9) If the monitoring camera 2 is provided 

with a device (not shown in the Figures) for reading 
and writing data in and from an external storage 
medium, such as a flexible disk, a CD-ROM or a DVD- 
ROM, a storage medium may be provided with for 
storing a program for causing the monitoring camera 2 
to function as the image rearranging unit 2071, the 
moving-object detector 2072, the mode switch 
controller 2073, the power supply controller 2074, 
the sensing controller 2075, the drive controller 
2076, the saved image generator 2077, the 
communication controller 2078 and the storage 2079, 
and the program is installed in the monitoring camera 
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2 such that the monitoring camera 2 may be provided 
with the functions of the image rearranging unit 2071 
and the other functional blocks and units. 
[0205] (10) Although the moving object is detected 

from the original image in the foregoing embodiment, 
the present invention is not limited thereto. A 
wide-angle image as shown in FIG. 10A may be 
generated also in the close-observation mode and a 
moving object may be detected from this wide-angle 
image . 

[0206] (11) Although the viewing direction of the 

camera 21 is changed in the panning direction and the 
tilting direction in the foregoing embodiment, the 
present invention is not limited thereto. The 
viewing direction of the camera 21 may be changed in 
parallel or translated by moving the camera 21 along 
a plurality of axes which intersect with each other. 
[0207] (12) An image magnified more than the 

close - observation image may be generated by applying 
digital zooming to the close- observation image, and 
this magnified image may be displayed on the display 
section 32 of the controller 3. 

[0208] In this case, the digitally zoomed image 

has a slightly lower resolution when being displayed. 
However, since the close - observation image has a high 
resolution, the image having a relatively high 
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resolution can be obtained even if digital zooming is 
applied at a relatively large zooming ratio. 
[0209] (13) In the close-observation mode, the 
rearranging process is carried out by extracting the 
pixels of the telephoto area portion of the original 
image which has a high resolution. Instead thereof, 
the image for the close -observation mode may be 
extracted by restricting the pho to - elect rically 
converted area on the image sensing section 202 by 
means of the a sensing controller 2075. 

[0210] As this invention may be embodied in several 
forms without departing from the spirit of essential 
characteristics thereof, the present embodiment is 
therefore illustrative and not restrictive, since the 
scope of the invention is defined by the appended 
claims rather than by the description preceding them, 
and all changes that fall within metes and bounds of 
the claims, or equivalence of such metes and bounds 
are therefore intended to embraced by the claims. 
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